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Introduction 

Throughout  the  Department  of 
Defense  (DoD),  analysts  use  Mod¬ 
els  and  Simulations  (M&S)  to  assist 
decision  makers  in  choosing  among  vari¬ 
ous  strategic  courses  of  action,  whether  or 
not  to  purchase  particular  weapons  sys¬ 
tems,  implement  programs,  and  so  on.  The 
cost  of  these  weapon  systems  and  pro¬ 
grams  can  run  into  billions  of  dollars,  and 
there  is  always  the  possibility  of  lives  lost. 
Therefore,  ensuring  that  our  models  are 
logically  coherent  and  provide  believable 
results  is  of  paramount  importance.  Mod¬ 
els  that  contain  non-monotonic  output  may 
be  unbelievable  to  decision-makers.  A 
model  is  non-monotonic  if  adding  capabili¬ 
ty  to  one  side,  while  holding  everything 
else  constant,  results  in  a  less  favorable 
outcome  for  that  side.  When  a  model  used 
to  decide  between  alternatives  exhibits 
non-monotonic  behavior,  the  model’s 
validity,  as  well  as  its  usefulness  for  analy¬ 
sis,  may  come  into  question. 

Non-monotonic  behavior  has  been 
observed  in  large,  well-established  deter¬ 
ministic  combat  models.  For  example, 
Saeger  and  Hinch,  in  a  force  thinning 
study  that  was  part  of  the  Quadrennial 
Defense  Review-1997,  found  that  the 
model  they  were  using  suggested  that  more 
capability  sometimes  resulted  in  worse  out¬ 
comes.1  This  article  summarizes  some  of 
the  findings  in  Vinyard  (2001)  of  high¬ 
dimensional  explorations  of  the  relatively 
simple  Dewar  model.2  In  particular,  two 
questions  are  addressed:  (1)  How  wide¬ 
spread  is  non-monotonicity  in  the  model? 
and  (2)  Can  the  response  surface  in  non¬ 
monotonic  regions  be  made  more 
amenable  to  interpretation  by  decision¬ 


makers  without  destroying  the  chaos  that 
may  be  inherent  to  both  real  combat  and 
the  model? 

Background 

In  1991,  Dewar  et  al.,  released  a  study 
showing  that  even  simple,  deterministic 
combat  models  can  be  subject  to  non¬ 
monotonic  behavior.3  This  finding  was  not 
new.  However,  the  finding  that  the  non¬ 
monotonicity  was  due  to  chaos  inherent  in 
the  model  was  new.  Perhaps  this  shouldn’t 
have  been  surprising,  as  combat  models  are 
nonlinear  systems  with  delayed  feedback 
processes  that  can  reinforce  or  dampen  the 
system.  That  is,  combat  models  are  exactly 
the  type  of  model  likely  to  exhibit  chaotic 
behavior.  Of  course,  this  can  be  troubling 
for  modelers  because  a  characteristic  of 
chaotic  systems  is  extreme  sensitivity  to 
initial  conditions.  Furthermore,  it  is  an 
inescapable  fact  that  no  matter  how  careful 
our  measurements,  the  data  used  in  our 
analyses  are  subject  to  errors.  In  chaotic 
systems,  even  if  the  magnitude  of  these 
errors  is  extremely  small,  the  uncertainty 
associated  with  the  errors  creates  uncer¬ 
tainty  about  our  knowledge  of  the  system 
in  the  future.  In  fact,  Sandmeyer,  found 
that  a  computer’s  double  precision  round¬ 
off  algorithm  qualitatively  affected  a  large, 
deterministic  combat  model’s  results.4 

Dewar  et  al.  cautioned  that  the  chaotic 
behavior  manifested  in  a  combat  model 
may  or  may  not  be  an  accurate  reflection  of 
chaos  on  the  battlefield.3  However,  one  of 
the  essential  aspects  of  combat  may  be 
chaos — for  want  of  a  nail ...  the  battle  was 
lost.  A  combat  model  that  exhibits  chaotic 
behavior  in  an  appropriate  way  seems,  on 
an  intuitive  level,  to  be  more  realistic  than 
a  model  that  does  not. 

Previous  Research  and 
Suggested  Remedies 

The  original  article  by  Dewar  et  al. 
spawned  a  series  of  articles.  Many  of  these 
papers  explored  ways  to  explain,  over¬ 
come,  or  attenuate  the  effects  of  chaos  and 
the  resulting  non-monotonicities.  Palmore 
offered  multiple  causes  of  instabilities  and 
discussed  several  ways  to  address  these 


problems.5  Louer  advocated  parametric 
variation  of  variables  and  stochastic  deci¬ 
sion  thresholds.6  He  claimed  that  these 
methods  cope  with  the  underlying  chaos, 
while  providing  results  that  are  informative 
and  trends  that  are  monotonic  in  behavior. 
Louer  also  suggested  the  use  of 

experimental  design  .  .  .  covering  the 
full  range  of  uncertainties  ...  to  develop 
distributions  of  the  response  functions.  . 
.  .  The  skilled  analyst  then  needs  to  .  .  . 
assess  if  they  have  any  significant  influ¬ 
ence  on  the  trends  of  the  response  func¬ 
tion  curves.7 

Cooper  urged  “far-ranging  sensitivity 
trials,  to  explore  more  of  a  model’s 
domains.”8  Cooper  also  showed  how  a 
minor  change  to  the  Dewar  model’s  deci¬ 
sion  rules  results  in  purely  monotonic 
behavior  in  the  two-dimensional  subspace 
he  examined.9  Huber  and  Tolk  showed 
that  “non-monotonic  effects  may  largely  be 
eliminated  if  dynamic  mission-oriented 
decision  mles  are  used  rather  than  the  static 
state-oriented  decision  thresholds  [Dewar 
used].”10  It  has  also  been  shown  that  care¬ 
ful  stochastic  manipulation  of  both  deci¬ 
sion  thresholds  and  attrition  coefficients 
can  significantly  smooth  the  non-monoto- 
nicities  that  arise  due  to  dynamic  instabili¬ 
ties  inherent  in  combat  models. 1 1 

Saeger  and  Hinch  recently  discovered 
non-monotonic  behavior  in  a  large,  deter¬ 
ministic  model.1  In  an  attempt  to  get 
meaningful  results  out  of  the  model,  they 
defined  a  neighborhood  of  the  phase  space, 
randomly  perturbed  selected  variables,  and 
attempted  to  fit  a  probability  curve  to  the 
output  values  of  multiple  mns.  They  con¬ 
cluded: 

[T]he  probability  distribution  of  the 
[response  curve]  is  a  function  of  the 
variable  that  is  perturbed.  [But  there  is] 
no  obvious  way  to  determine,  a  priori, 
which  variables  to  perturb  [nor]  by  what 
magnitude  to  perturb  them. 

Their  recommendation  was  to  “perturb 
all  variables  [and]  perform  sensitivity 
analysis  in  perturbation  magnitude.” 

(See  COMBAT  MODELS,  p.  36) 
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Figure  1.  The  Response  Surface  of  the  Original  Dewar  Model. 


COMBAT  MODELS 

(continued  from  p.  19) 

Searching  for  Non-Monotonicity 
in  the  Dewar  Model 

The  previously  published  papers  on  the 
Dewar  model  examined  only  one  or  two  of 
the  model’s  18  dimensions.  Thus,  the  ques¬ 
tion  must  be  asked:  Are  these  findings 
anomalies  that  occur  only  in  the  small  por¬ 
tions  of  the  model  that  have  been  examined, 
or  do  they  generalize  to  other  measures  and 
dimensions?  The  combinatorial  possibili¬ 
ties  of  main  effects  and  interactions  among 
the  1 8  dimensions  are  too  great  to  examine 
en  masse.  Consequently,  we  use  advanced 
statistical  designs  and  billions  of  computa¬ 
tional  experiments  to  more  fully  explore  the 
model. 

The  original  Dewar  model  is  a  determin¬ 
istic  time-step  simulation  of  a  homogeneous 
Lanchester  square  law  battle  (see  [3]  for 
details).  In  addition  to  the  attrition  rate 
coefficients,  the  model  has  parameters  for 
initial  force  sizes,  reserves,  reinforcement 
levels,  reinforcement  delays,  and  decision 
thresholds.  At  each  time  step,  depending  on 
the  engaged  force  ratio  and  force  levels, 
each  side  makes  decisions  on  whether  to 
withdraw  or  call  in  reinforcements.  There  is 
a  natural  symmetry  in  the  parameters — i.e., 
for  each  Blue  parameter  there  is  a  corre¬ 
sponding  Red  parameter.  The  previous 
research  focused  on  the  initial  forces  sub¬ 
space  (i.e.,  the  two-dimensional  space 
defined  by  initial  Blue  force  level  and  initial 
Red  force  level)  and  the  binary  outcome 
measure  ‘who  wins.’  Figure  1  shows  the 
non-monotonic  output  of  this  subspace  in 
the  Dewar  model.  In  this  two-dimensional 
graph,  initial  Red  force  levels  vary  from  ten 
to  3500,  in  increments  of  ten.  Initial  Blue 
force  levels  vary  from  ten  to  2000,  also  in 
increments  of  ten.  Thus,  the  model  was  mn 
69,451  times  to  generate  this  surface.  The 
black  region  represents  those  initial  force 
levels  that  result  in  a  Red  win.  Consider  the 
following  two  scenarios: 

a.  Initial  Blue  forces  are  fixed  at  900  (this  is 
indicated  by  a  line  on  the  graph),  and  ini¬ 
tial  Red  forces  vary  from  ten  through 
3500.  As  additional  troops  are  added  to 
the  initial  Red  force,  with  all  other  vari¬ 
ables  held  constant,  the  trend  is  monoton¬ 
ic.  At  an  initial  force  level  of  about  2800 
troops,  Red  starts  to  win.  After  that, 
adding  troops  to  Red’s  initial  force  level 
results  in  continued  Red  wins,  just  as  one 


would  expect. 

b.  The  initial  Blue  force  level  is  fixed  at  450, 
while  Red  varies  from  700  to  1800. 
Now,  the  response  trend  goes  from  Blue 
wins  to  Red  wins,  and  back  and  forth 
many  times.  This  non-monotonic  trend 
seems  to  make  it  impossible  for  a  deci¬ 
sion-maker  to  decide  whether  or  not 
adding  more  Red  Forces  is  a  good  idea. 

Note  the  broad  region  of  extreme  non¬ 
monotonicity  in  Figure  1 .  Also  note  that 
large  portions  of  this  subspace  contain  nice 
monotonic  regions.  If  this  graph  is  any  indi¬ 
cation  of  the  subspaces  that  exist  in  larger 
models,  then  it  is  easy  to  see  why  extreme 
non-monotonicity  might  go  unnoticed,  even 
when  it  exists.  In  larger  models,  the  dimen¬ 
sionality  of  the  phase  space  is  incomprehen¬ 
sibly  vast.  It  is  entirely  possible  that  these 
large  models  are  operating  in  purely  monot¬ 
onic  regions.  However,  it  is  also  possible 
that  they  are  teetering  on  the  edges  of  non¬ 
monotonic  regions  like  the  one  pictured 
here.  ,  , 

In  the  Dewar  model,  there  are  1^1  = 
153  pairs  of  variables.  We  chose  to  search 
the  subspaces  associated  with  the  nine  nat¬ 
ural  pairs  of  variables,  with  a  natural  pairing 
consisting  of  the  same  parameter  for  Red 
and  Blue.  For  each  of  these  pairs,  we  want 


to  see  if  the  surface  is  monotonic  over  a 
range  of  settings  for  the  other  16  parame¬ 
ters.  To  do  this,  we  used  random  Latin 
Hypercube  Sampling  on  the  16  parameters. 
From  McKay  et  al.,  Latin  Hypercube  Sam¬ 
pling  “can  be  viewed  as  a  K-dimensional 
extension  of  Latin  Square  Sampling”  and 
generates  an  efficient  “space-filling” 
design.12 

For  each  of  the  nine  natural  two-dimen¬ 
sional  subspaces,  16  surfaces  are  generated 
and  assessed  for  non-monotonicity.  In  total, 
9*16*69,451  =  10,000,944  battles  are  simu¬ 
lated  to  generate  144  surfaces.  In  designing 
our  sample,  we  adhere  to  the  original  Dewar 
model’s  basic  structure  of  a  smaller,  more 
efficient  force  opposing  a  larger,  less  effec¬ 
tive  force;  or,  if  you  prefer,  a  smaller  Blue 
defensive  force  opposing  a  larger  Red 
attacking  force.  To  preserve  the  original 
model’s  tension  between  opposing  forces, 
we  restrict  the  domain  of  the  remaining 
variables  to  fairly  thin  hyperplanes,  centered 
at  the  nominal  values  of  the  original  model. 

This  exploration  found  that  non-monoto- 
nicity,  with  respect  to  the  measure  ‘who 
wins,’  is  prevalent  in  the  model,  with  non¬ 
monotonicity  for  ‘who  wins’  being  found  in 
seven  of  the  nine  two-dimensional  sub¬ 
spaces  explored  (see  Table  1).  In  fact,  five 
of  the  subspaces  exhibit  pervasive  non- 
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Two-dimensional  Percent  of  Response  Surfaces 

Dewar  Model  Subspaces  Containing  Non-monotonic  Behavior 


Initial  Force  Levels  100% 

Force  Ratio  Reinforcement  Thresholds  8 1  % 

Percent  of  Remaining  Forces  Reinforcement  Threshold  19% 

Force  Ratio  Withdrawal  Threshold  0% 

Percent  of  Remaining  Forces  Withdrawal  Threshold  0% 

Reinforcement  Blocks  Available  6% 

Reinforcement  Block  Delay  100% 

Reinforcement  Block  Size  88% 

Attrition  Coefficients  94% 


Table  1.  Latin  Hypercube  Sampling  Results:  78  of  the  144  response  surfaces  found 
during  the  Latin  Hypercube  Sampling  exhibited  non-monotonic  behavior. 


monotonicity,  with  it  showing  up  in  over 
80  percent  of  the  surfaces  checked.  In  total, 
54%  of  the  surfaces  generated  contain  non¬ 
monotonic  regions. 

Figure  2  shows  examples  of  the  newly 
discovered  widespread  non-monotonicity 
in  the  Dewar  model.  The  top  row  of  the 
figure  displays  two  of  the  many  striking 


examples  we  discovered  using  Latin 
Hypercube  Sampling.  These  two  graphs 
exhibit  non-monotonicity  with  respect  to 
the  MOE  ‘length  of  battle.’  The  two  bot¬ 
tom  graphs  in  Figure  2  exhibit  non-monot- 
onicity  with  respect  to  the  MOE  ‘who 
wins’  in  the  force  ratio  reinforcement  and 
reinforcement  block  size  subspaces. 


Mitigating  Non-Monotonicity 
in  the  Dewar  Model 

The  fact  that  so  many  subspaces  contain 
non-monotonic  behavior  is  cause  for  con¬ 
cern.  The  Dewar  model  contains  some  of 
the  same  basic  processes  that  many  of  the 
larger  models  use,  such  as  decision  thresh¬ 
olds  and  attrition  processes.  If  the  interac¬ 
tion  of  these  processes  in  the  Dewar  model 
generates  such  widespread  non-monotonic 
behavior,  then  the  larger,  more  complex 
models  may  also  be  affected  by  similar  non¬ 
monotonicities.  What  can  be  done  to  gener¬ 
ate  interpretable  responses  despite  the  non¬ 
monotonicities  caused  by  the  chaos  in  the 
battle  trace?  Much  of  the  literature 
reviewed  for  this  paper  indicates  that  sto¬ 
chastic  modeling  can  be  a  useful  way  to 
deal  with  non-monotonic  behavior  in  both 
the  simple  Dewar  model  and  other,  more 
complex  models. 

To  examine  how  making  parameters  sto¬ 
chastic  affects  non-monotonicity  in  the 
Dewar  model,  we  ran  a  fractional  factorial 
experiment  to  determine  the  effect  of  sto¬ 
chastic  modeling  on  the  trends  of  the 
response  surface.  The  experiment  varied 
nine  factors  consisting  of  the  nine  types  of 
parameters  in  the  model.  Each  factor  had 
two  levels,  deterministic  and  stochastic.  To 
efficiently  search  all  nine  factors  simultane¬ 
ously,  a  29-3,  resolution  V,  fractional  factori¬ 
al  design  is  used.13  This  requires  64  differ¬ 
ent  input  settings.  Each  surface  consists  of 
69,45 1  points.  Plus,  each  point  in  a  stochas¬ 
tic  model  must  be  estimated.  To  get  precise 
estimates,  we  use  1000  replications  to  esti¬ 
mate  the  probability  that  Red  wins  at  each 
point.  Thus,  about  4.5  billion  battles  were 
simulated  in  the  fractional  factorial  experi¬ 
ment. 

It  is  not  always  easy  to  distinguish 
between  bona  fide  non-monotonicities  and 
random  variation  over  an  entire  surface.  Six 
parameters — measuring  the  number  of  non¬ 
monotonic  jumps,  trends,  statistically  signif¬ 
icant  jumps,  statistically  significant  trends, 
and  functions  thereof — were  used  to  assess 
the  extent  of  the  non-monotonicities.  See 
[2]  for  how  this  was  done  and  an  empirical 
assessment.  Following  previous  analyses, 
the  attrition  coefficients  are  modeled  as  nor¬ 
mal  random  variables,  with  means  equal  to 
the  nominal  Dewar  model  values  and  stan¬ 
dard  deviations  equal  to  ten  percent  of  the 
square  root  of  their  means.  All  other  para¬ 
meters,  when  stochastic,  are  modeled  as 

(See  COMBAT  MODELS,  p.  38) 
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Figure  2.  Four  of  the  Many  Two-dimensional  Subspaces 
Exhibiting  Non-monotonic  Behavior 
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COMBAT  MODELS 

(continued from  p.  37) 

uniform  random  variables  with  intervals, 
centered  on  the  nominal  Dewar  model  val¬ 
ues,  ranging  from  plus  to  minus  five  percent 
of  the  nominal  value.  These  small  random 
deviations  are  within  the  uncertainties  that 
would  be  present  on  a  real  battlefield. 

The  results  are  dramatic  and  convincing, 
and  support  previous  work.1’6’7’11  Stochas¬ 
tic  perturbation  usually  dramatically  reduces 
the  non-monotonic  behavior  of  the  response 
surface,  but  can,  by  some  measures,  exacer¬ 
bate  it.  The  attrition  coefficients  are  the 
model  parameters,  over  the  values  we  inves¬ 
tigated,  that  have  the  greatest  effect  on  the 
reduction  of  the  non-monotonic  behavior — 
see  [2]  for  a  detailed  analysis  of  the  effects 
and  interactions  between  parameters.  Fig¬ 
ure  3  shows  the  same  surface  as  Figure  1, 
with  all  of  the  parameters  stochastic.  Unlike 
the  previous  graph,  Figure  3  appeals  to  our 
intuition(the  outcome  remains  uncertain 
until  one  side  or  the  other  quits  the  field  of 
battle. 

Conclusions 

Caveat  actor  et  cavendo  tutus 

Feigenbaum,  Mandlebrot  and  other  pio¬ 
neers  of  chaos  theory  have  shown  that  chaos 
is  the  rule  rather  than  the  exception  in  the 
real  world.  14  The  possibility  that  chaos  and 
its  consequent  non-monotonicity  are  also 
the  mle  in  large,  complex  combat  models  is 
very  real.  The  Dewar  model  is  relatively 
small  compared  to  most  of  the  larger,  more 
complex  models  that  DoD  currently  uses. 
Nonetheless,  it  includes  many  of  the  same 
processes  present  in  these  much  larger  mod¬ 
els.  Thus,  it  is  reasonable  to  suppose  that 
what  we  learn  from  studying  the  Dewar 
model  will  help  us  with  our  larger  models. 
The  bottom  line  is  that  non-monotonicity 
may  be  more  pervasive  in  combat  models 
than  previously  suspected.  Stochastic  mod¬ 
eling  can  be  a  viable  method  for  dealing 
with  non-monotonic  response  surfaces. 
Flowever,  stochastic  modeling  must  be  done 
carefully.  When  it  is,  the  non-monotonic 
behavior  of  the  model  can  be  dramatically 
reduced  while  maintaining  the  chaos  that 
may  be  inherent  to  combat,  thereby  making 
the  trends  of  the  response  surface  more  use¬ 
ful  for  comparative  analyses.  Better  analy¬ 
ses  can  help  decision-makers  save  time, 
money,  and  other  assets(and,  perhaps, 
another  soldier,  sailor,  airman  or  Marine 
will  get  to  enjoy  their  pension. 


Figure  3.  When  all  parameters  are  stochastic,  the  response  surface  represents  the 
probability  of  a  Red  win.  In  this  graph,  the  regions  where  Red  or  Blue  wins  are 
clearly  delineated.  The  gray  region  represents  those  initial  force  levels  where 
the  outcome  is  uncertain. 
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